Mutation Testing in Evolving Systems: Studying the Relevance of Mutants to Code Evolution

Context: When software evolves, opportunities for introducing faults appear. Therefore, it is important to test the evolved program behaviors during each evolution cycle. However, while its complexity also evolving, challenges testing process. To deal with this issue, techniques should be adapted target effect of changes instead entire functionality. end, commit-aware mutation , a powerful technique, has been proposed. Unfortunately, challenging due complex semantics involved. Hence, pertinent understand characteristics, predictability, and potential technique. Objective: We conduct an exploratory study investigate properties commit-relevant mutants i.e., elements testing, by proposing general definition experimental approach identify them. thus aim at investigating prevalence, location, comparative advantages over time (i.e., evolution). predictive power several commit-related features in identifying selecting essential best-effort application case. Method: Our relies on notion observational slicing, approximated higher-order mutation. Specifically, our utilizes impact mutants, effects one mutant another capturing analyzing implicit interactions between changed unchanged code parts. The analyses millions (over 10 million), 288 commits, five (5) different open-source projects involving 68,213 CPU days computation sets ground truth where we perform analysis. Results: analysis shows that are located mainly outside commit change (81%), suggesting limitation previous work. note effective selection reducing number up 93%. In addition, demonstrate relevant significantly more efficient than state-of-the-art baselines, random only within change. (e.g., change, type, size) predicting found most proxy do not reliably predict . Conclusion: This empirical highlights demonstrates importance when evolving systems.